Comparing patterns

ABSTRACT

A first image (or other pattern) is represented by a first ordered set of elements A each having a value and a second pattern is represented by a second such set. A comparison of the two involves performing, for each of a plurality of elements x of the first ordered set the steps of selecting from the first ordered set a plurality of elements x′ in the vicinity of the element x under consideration, selecting an element y of the second ordered set and comparing the elements x′ of the first ordered set with elements y′ of the second ordered set (each of which has the same position relative to the selected element y′ of the second ordered set as a respective one x′ of the selected plurality of elements of the first ordered set has relative to the element x under consideration). The comparison itself comprises comparing the value of each of the selected plurality of elements x′ of the first set with the value of the correspondingly positioned element y′ of the like plurality of elements of the second set in accordance with a predetermined match criterion to produce a decision that the plurality of elements of the first ordered set matches the plurality of elements of the second ordered set. The comparison is them repeated with a fresh selection of the plurality of elements x′ of the first set and/or a fresh selection of an element y of the second ordered set generating a similarity measure V as a function of the number of matches. Preferably, following a comparison resulting in a match decision, the next comparison is performed with a fresh selection of the plurality of elements x′ of the first set and the same selection of an element y of the second set.

This application is the U.S. national phase of international applicationPCT/GB03/01209 filed 21 Mar. 2003 which designated the U.S. and claimsbenefit of EP 02252097.7, dated 22 Mar. 2002, the entire content ofwhich is hereby incorporated by reference.

BACKGROUND

1. Technical Field

The present invention is concerned with comparing one pattern withanother, and is of particular interest in the comparison oftwo-dimensional patterns such as visual images, although applicable alsoto one-dimensional patterns and patterns having three or moredimensions.

2. Related Art

Standard approaches to pattern recognition use templates to recognizeand categorize patterns [1]. Such templates take many forms, but theyare normally produced by a statistical analysis of training data andmatched with unseen data using a similarity measure [2]. The statisticalanalysis is normally carried out over a number of intuitively selectedfeatures that appear to satisfy the needs of the recognition task. Forexample, in speech recognition, templates can be encapsulated as HiddenMarkov Models derived in the frequency domain and, in Optical CharacterRecognition, the templates take the form of the character fontsthemselves. In the case of face recognition, a number of intuitivelychosen features such as skin texture, skin color and facial featureregistration are used to define face templates. In a CCTV surveillanceapplication, intruders are normally detected through a process of framesubtraction and background template modelling which detects movement andremoves background effects from the processing [3]. In many cases, thenumber of features leads to a computationally unmanageable process andPrincipal Components Analysis and other techniques are used to scaledown the problem without significantly reducing performance (e.g., seepartek.com website/index.html). These approaches achieve great successin non-noisy environments, but fail when the pattern variability andnumber of pattern classes increase.

Some techniques for analysis of images or other patterns where thepattern is compared with other parts of the same pattern are describedin our earlier patent applications as follows.

European patent application 00301262.2 (publication No. 1126411);

International patent application PCT/GB01/00504 (publication No. WO01/61648);

International patent application PCT/GB01/03802 (publication No. WO02/21446);

U.S. patent application Ser. No. 977,263/09 filed 16 Oct. 2001(publication No. 20020081033);

-   -   as well as the following papers published by the inventor:

-   Stentiford F W M, “An estimator for visual attention through    competitive novelty with application to image compression”, Proc.    Picture Coding Symposium 2001, Seoul, 25-27 April, pp 101-104, 2001.

-   Stentiford F W M, “An evolutionary programming approach to the    simulation of visual attention”, Proc. Congress on Evolutionary    Computation 2001, Seoul, pp 851-858, 27-30 May, 2001.

SUMMARY

According to one aspect of the present invention there is provided amethod of comparing a first pattern represented by a first ordered setof elements each having a value with a second pattern represented by asecond ordered set of element each having a value, comprisingperforming, for each of a plurality of elements of the first ordered setthe steps of:

(i) selecting from the first ordered set a plurality of elements in thevicinity of the element under consideration, the selected elementshaving, within the ordered set, respective mutually different positionsrelative to the element under consideration;

(ii) selecting an element of the second ordered set;

(iii) comparing the selected plurality of elements of the first orderedset with a like plurality of elements of the second ordered set each ofwhich has within the second ordered set the same position relative tothe selected element of the second ordered set as a respective one ofthe selected plurality of elements of the first ordered set has relativeto the element under consideration, said comparison comprising comparingthe value of each of the selected plurality of elements of the firstordered set with the value of the correspondingly positioned element ofthe like plurality of elements of the second ordered set in accordancewith a predetermined match criterion to produce a decision that theplurality of elements of the first ordered set matches the plurality ofelements of the second ordered set;

(iv) repeating said comparison with a fresh selection of the pluralityof elements of the first ordered set and/or a fresh selection of anelement of the second ordered set; and

(v) generating for the element under consideration a similarity measureas a function of the number of comparisons for which the comparisonindicates a match.

Other aspects of the invention are defined in the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments of the present invention will now be described withreference to the accompanying drawings, in which:

FIG. 1 is a block diagram of an apparatus for performing the invention;

FIG. 2 is a diagram illustrating operation of the invention;

FIG. 3 is a flowchart of the steps to be performed by the apparatus ofFIG. 1 in accordance with one embodiment of the invention; and

FIGS. 4 to 9 illustrate some images and numerical results obtained forthem

FIG. 1 shows an apparatus consisting of a general purpose computerprogrammed to perform image analysis according to a first embodiment ofthe invention. It has a bus 1, to which are connected a centralprocessing unit 2, a visual display 3, a keyboard 4, a scanner 5 (orother device, not shown) for input of images, and a memory 6.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

In the memory 6 are stored an operating system 601, a program 602 forperforming the image analysis, and storage areas 603, 604 for storingtwo images, referred to as image A and image B. Each image is stored asa two-dimensional array of values, each value representing thebrightness of a picture element within the array.

The image arrays are shown schematically in FIGS. 2 a and 2 b. Image Aconsists of a 20×20 array of picture elements x=(x₁, x₂) where x₁ and x₂are the horizontal and vertical positions of the elements within theimage. Each element has a brightness represented by a respective value a=a (x). Similarly, the image B consists of a 20×20 array of pictureelements y=(y₁, y₂) having brightness values b.

A method of image analysis performed by the program 602 is performed inaccordance with the flowchart shown in FIG. 3. It has as its aim togenerate a measure V of similarity between the images; morespecifically, a high value of V indicates that image B contains visualmaterial similar to the content of image A.

It is assumed that the images have width xmax₁ and ymax₁ and heightxmax₂ and ymax₂, and the picture element co-ordinates are x₁=0 . . .xmax₁−1, x₂=0 . . . xmax₂−1.

In Step 100, a picture element x in image A is selected. They can bedealt with in any order, but it is convenient to select initially theelement x=(ε,ε) and on subsequent iterations, successive elements inraster-scan fashion, until all have been dealt with, except for thosewithin ε of the edge of the image i.e. up to (xmax₁−ε−1, xmax₂−ε−1).

In Step 101, the score V and a counter tries are set to zero.

In Step 102, an element y=(y₁, y₂) is chosen at random from image B,within a maximum distance s of the position of x, that is|x _(i) −y _(i) |≦s for all i

This limitation, which is optional, is represented in FIG. 2 b by thedotted square S. It improves efficiency if it is known thatcorresponding objects in the two images will not be shifted in positionby more than s picture elements in either co-ordinate direction: ineffect, s represents the maximum misregistration or local distortionbetween the two images.

The selection of y is also subject to the restriction that it is locatedno more than ε from the edge of the image, i.e.ε≦y _(i) ≦ymax_(i)−ε−1 for i=1,2

where ymax_(i) is the width or height of the image in picture elements(20 in this example).

In Step 103, a set S_(x) of picture elements x′, are selected, comparingthe element plus at least one further element, m in number, randomlyselected from image A, in the neighbourhood N of picture element x,where the neighbourhood N is a 2ε+1×2ε+1 square centred on x. That is,an element x′=(x₁′, x′₂) lies within N if|x _(i) ′−x _(i)|<ε for all i

We prefer that the neighbourhood set S_(x) includes the element xitself, although this is not actually essential. A neighbourhood setS_(y) is then defined in image B, with m+1 elements y′=(y₁′, y₂′) eachof which has the same position relative to element y as thecorresponding element x′ of S_(x) has from x, that isy _(i) ′−y _(i) =x _(i) ′−x _(i) for all i

(The reason for the edge restrictions mentioned above is of course toavoid choosing a neighbourhood which extends outside the image area). InStep 104, the counter tries is incremented, and in Step 105 its value ischecked so see whether sufficient comparisons have been performed.

Assuming for the present that this is not the case, then at Step 106 thetwo sets S_(x), S_(y) are compared. If the difference between the valueof an element x′ of the set S_(x) and the value of the correspondinglypositioned element x′ of the set S_(y) is less than a threshold δ, i.e.|a( x ′)−b( y ′)|<δ

then the two elements are considered to match. The two sets areconsidered to match only if every element x′ of the set S_(x) matchesthe corresponding element y′ of the set S_(y).

If the sets do not match, the element y is discarded as a candidate forfurther comparisons and the process resumes at step 102 with a freshpoint y being chosen. If on the other hand, they match, then y isretained for a further iteration. Ignoring Step 107, for the present,the process proceeds with step 108 where the score V is incremented.Different neighbourhood sets are then generated at Step 103 and theprocess repeated from that point.

When, at Step 105, the count tries of the number of comparisons exceedsa threshold, the process for the current x is terminated: at Step 109the value of V is stored in the memory 6 and then, at step 110, a checkis performed as to whether all points x have been dealt with. If so,then firstly the average score Va (being the average of all theindividual scores V for the picture elements x in image A) is calculatedat Step 111 and the process terminated at 112; otherwise the wholeprocess is repeated from Step 100 for a fresh point x.

The threshold value used at Step 105 could be set to a desired number ofcomparisons t, and Step 107 omitted. In practice, however, we find thatthe early comparisons often do not find a match, but once a match isfound a significant number of different neighbourhood sets show matcheswith the same y. In the interests of providing a score V which is easilyinterpreted, the first few iterations are not scored; the process asshown includes a test 107 which prevents the score being incrementedduring the first init comparisons. The threshold used at Step 105 isthus increased by init so that the maximum potential score is t. In thisway, we find that the chances of obtaining a high or maximum score forimages which are very similar or identical is greatly increased.

Specific values for the various parameter used in tests were as follows:

-   Image A size xmax₁×xmax₂: 20×20-   Image B size xmax₁×xmax₂: 20×20-   Neighbourhood parameter ε: 4-   Neighbourhood set size m: 3-   Maximum A−B shift s: 7-   Number of comparisons t: 50-   Initial comparisons init: 20

Naturally, for higher definition images, correspondingly larger valuesof ε and s would be chosen. The values of t and init have to beincreased as s increases in order to obtain equally reliable andstatistically significant results.

The results of these tests are given later; where the parameters useddiffer from those quoted above, this is noted.

It will be seen that the score of a picture element x in the pattern Ais high if many randomly selected S_(x) match S_(y) for a given y inpattern B. It is worth noting that the Visual Attention score (discussedin our earlier patent applications) of picture element x in pattern A ishigh if S_(x) mis-matches S_(y) where y is not held but is randomlyselected from pattern A. Whereas Visual Attention requires no memorysave the single image A, the present method requires also theinformation contained in pattern B to detect commonality.

A location x will be worthy of cognitive attention if a sequence of tneighbourhood sets S_(x) matches a high proportion of the sameneighbourhoods surrounding some y in pattern B. In FIG. 2, m=3 pictureelements x′ are selected in the neighbourhood of a picture element x inpattern A and matched with 3 picture elements in the neighbourhood ofpicture element y in pattern B. Each of the picture elements mightpossess three colour intensities, so a=(a_(r),a_(g),a_(b)) and theneighbourhood set of the second picture element y matches the first ifthe colour intensities of all m +1 corresponding picture element havevalues within δ of each other. Picture elements x in A that achievelarge numbers of matches over a range of t neighbourhood sets S_(x) withneighbourhood sets S_(y) around y in B are assigned a high score. Thismeans that neighbourhood sets in A possessing structure present in Bwill be assigned high scores.

Some image analysis techniques carry out comparison calculations betweenimages using patches that are neighbourhoods in which all the pictureelements are employed. Patches match when a measure of correlationexceeds a certain threshold. This approach is unable to make best use ofdetail that is smaller than the size of the patch except in the case inwhich the correlation measure is designed to identify a specifictexture. The random picture element neighbourhoods S_(x) used in thismethod do not suffer from this disadvantage.

The gain of the scoring mechanism is increased significantly byretaining the picture element location y if a match is detected, andre-using y for comparison with the next of the t neighbourhood sets. Itis likely that if a matching picture element configuration is generated,other configurations will match again at the same point, and thislocation y once found and re-used, will accelerate the rise of the scoreprovided that the sequence is not subsequently interrupted by amismatch.

If, however, S_(x) subsequently mis-matches at that location, the scoreis not incremented, and an entirely new location y in pattern B israndomly selected ready for the next comparison. In this way competinglocations in pattern B are selected against if they contain littlecommonality with the neighbourhood of x in pattern A.

Some possible variations will now be discussed.

a) It is not essential that all picture elements x be analysed: thus, ifdesired, a subset—perhaps on a regular grid—could be chosen.

b) The above method assumes single (brightness) values for the pictureelements. In the case of colour images, the images could firstly beconverted to monochrome. Alternatively, if each picture element isexpressed as three values, such as red, green and blue, or luminanceplus colour difference, then the test for a match would involveconsideration of the distance between the element values in 3dimensions. For example, if the colour component values are a=(a_(r),a_(g), a_(b)), b=(b_(r), b_(g), b_(b)) for the two image then thecriterion for a match might be (a_(r)−b_(r))<δ and |a_(g)−b_(g)|<δ and|a_(b)−b_(b)|<δ. See below for further discussion of distance measures.

c) The strategy of choosing y at random could be modified by providingthat when a element x=(x₁, x₂) produces a high score V against elementy, and an adjacent element x (e.g. (x₁+1, x₂)) is to be dealt with, thenthe first element y to be selected in image B could be thecorrespondingly adjacent element (i.e. (y₁+1, y₂)). Further selectionsof y would be random, as before. In the event that the high score forelement x was produced as a result of comparisons with more than one y,then one would choose the element adjacent to whichever y made thelargest contribution to the high score. To implement this is would benecessary to track, temporarily, the coordinates of the differentpicture elements y and their partial scores.

d) The process need not necessarily be carried out for the whole image.For example, if a region of image A has been identified as being ofspecial interest—perhaps using the method described in one of ourearlier patent applications referred to above—then the picture elementsx dealt with may be just those lying within the identified region.

References here to random selection include the possibility of selectionusing a pseudo-random process.

The method is not applicable solely to images, or even three-dimensionalpatterns, but can be used for any ordered set of values in 1, 2, 3 ormore dimensions.

More generally, suppose that one wishes to generate a measure of thesimilarity between two patterns A and B, where both the patterns and theindividual pattern of values within a pattern may have 1, 2 or moredimensions.

Let a set of measurements a on pattern A correspond to a location x in Ain bounded n-space (x₁, x₂, x₃, . . . , x_(n)) wherex=(x₁,x₂,x₃, . . . , x_(n)) and a=(a₁,a₂,a₃, . . . , a_(p))

Define a function F such that a=F(x) wherever a exists. It is importantto note that no assumptions are made about the nature of F e.g.continuity. It is assumed that x exists if a exists.

Consider a neighbourhood N of x where{x′∈N iff|x_(i)−x′_(i)|<ε_(i) ∀i}

Select a set of m+1 random points S_(x) in N whereS_(x)={x′₁, x′₂, x′₃, . . . , x′_(m)} and F(x′_(i)) is defined.

-   -   where however one of the x′ is preferably constrained to equal        x. It is not found in practice to be necessary to constrain the        random selection to avoid selection of the same point twice:        i.e. the x′_(i) need not be distinct. Indeed, the x′_(i) could        all be coincident with x or x′₁.

Select a location y corresponding to the set of measurements b onpattern B for which F is defined.

Define the set S_(y)={y′₁, y′₂, y′₃, . . . , y′_(m)} wherex−x′ _(i) =y−y′ _(i) and F ( y′ _(i)) exists.

The neighbourhood set S_(x) of x is said to match that of y if|F _(j)( x )−F _(j)( y )|<δ_(j) and |F _(j)( x′ _(i))−F _(j)( y′_(i))|<δ_(j) ∀i,j.

In general δ_(j) is not a constant and will be dependent upon themeasurements under comparison i.e.δ_(j=f) _(j)(F(x),F(y)) j=1 . . . p

Note that for some problems rather than defining a match as requiringthe individual components a_(j)=F_(j)( ) and b_(j)=F_(j)( ) to match onemight prefer to apply a threshold to the vector distance between a and bsuch as the city block distance.

$\sum\limits_{j}{{a_{j} - b_{j}}}$

or the Euclidean distance

$\sqrt{\left\{ {\sum\limits_{j}\left( {a_{j} - b_{j}} \right)^{2}} \right\}}$

The operation of the method described in the flowchart of FIG. 3 will beillustrated by the following test results.

In the tests, 20×20 black and white images were used (i.e. the values ofa and b were always either 0 or 1). The tests however differ from theearlier description in that only black picture elements are scored—thatis, the match criterion is not merely that |a−b|<δ but that a=b=1.

In each case the accompanying Figure shows a first image A, plus severalsecond images B with which the first was compared, marked B1, B2 etc.(in once case two first images A1, A2 are compared with one image B).The coordinates x₁, y₁ 30are measured from left to right and thecoordinates x₂, y₂ from top to bottom. Each of the second images ismarked with the average score Va. Some Figures show a three-dimensionalplot of the individual scores V over the image area: in each case theright-hand axis represents the horizontal axis of the mark and the frontaxis represents the vertical axis of the image. The vertical axis of theplot represents the value of V (in the embedded recognition case ofExample 2, and 50−V in the other cases).

Example 1 Optical Character Recognition (FIG. 4)

An image A of the letter B is compared with images B1-B5, of the lettersA, B, C, D and E. The rank ordered average scores Va correspond to B, D,E, C, A. The results were generated with t=50, s=3 and ε=2.

Example 2 Embedded Recognition (FIG. 5)

An image A of an ‘X’ shape is compared with another image B1 containing9 smaller shapes including an ‘X’. The ‘X’ shape is again compared withimage B2 with the same 9 shapes except for the small ‘X’ which isaltered. The average scores V_(a) are 27.5 and 11.9 respectively,indicating the presence of ‘X’ features in the first and less in thesecond. The individual scores V for the respective comparisons are shownin the charts. The results were generated with t=50, s=7 and ε=2. A morecomplicated example of embedded recognition is the identification ofknown faces in a crowd of people.

Example 3 Trade Mark Retrieval (FIG. 6)

An image A of a special cross shape is first compared with 4 basicshapes B1-B4 including a large cross, each basic pattern representingclusters of similar shapes with high scores between each other. Thelarge cross generates the largest average score V_(a) (43.6). The samespecial cross in then compared with 4 varieties of cross B5-B8 includinga small version of itself (B5) which obtains a maximum score of 50. Thechart provides the values of (50−V) for each picture element in thecomparison with the large cross—the tall columns indicate the disparitywith the ends of the lower legs of the cross. The results were generatedwith t=50, s=7 and ε=2.

Example 4 Fingerprint Recognition (FIG. 7)

More complex patterns, some possessing a great deal of similar material,are able to be distinguished and retrieved as in the trade mark example.A simplified arch A matches best with a representative arch B4 (49.3),and then with an identical pattern (49.9) amongst other arches. Thechart provides the values of (50−V) for each picture element in thecomparison with the representative arch pattern—the tall columnsindicate the disparity with the ridge ends and on the border. Theresults were generated with t=50, s=3 and ε=2.

Example 5 Face Recognition (FIG. 8)

Although the data is extremely simplified, this example illustrates animplementation of face recognition. A face A is matched with 4representative faces B1-B4 and then with 4 faces in a sub-cluster B5-B8represented by the best fitting representative face B2 (45.7). The bestfit is with a non-identical face B7 in the sub-cluster (49.6). The chartprovides the values of (50−V) for each picture element in the comparisonwith the 2^(nd) closest pattern B6 (48.2)—the tall columns indicate thedisparity in the mouth region. The results were generated with t=50, s=3and ε=2.

Example 6 Surveillance (FIG. 9)

This example illustrates how intruders might be detected against avariable background. A pattern A1 representing a moving cloud, tree andfence obtains a high average score V_(a) (48.0) against a standardbackground B, but a version (A2) with an intruder present obtains a muchlower score (33.1). The chart provides the values of (50−V) for eachpicture element in the intruder image (33.1)—the tallest columnsindicate the location of the intruder. The results were generated witht=50, s=3 and ε=2.

This method carries out an analysis of two images that identifiesperceptually significant common features and differences without beingaffected by scale differences, and by local relative distortions andtranslations. This means that it can be applied to recognition problemsin which the object to be recognised is embedded in another image. Italso means that images may be categorised according to the measure offeature similarity with each other and these values used to define amulti-class recognition system and a Query By Example retrieval system.By the same token the method may be used to detect dissimilarities as inthe case of intruder detection or forgery detection. It may be used todetect disparities between images to detect motion or parallax. It doesnot necessarily rely upon prior knowledge of the content of the imagesand no training process is necessary. The algorithm is eminentlysuitable for parallel implementation.

Various advantages of the methods we have described will now bediscussed.

Pattern Recognition

Standard approaches to pattern recognition require large numbers ofrepresentative patterns from all classes in order to satisfy therequirements of the training process for the classifier. Suchclassifiers will still fail if the features selected do not characteriseunseen patterns not represented properly in the training set. Thismethod requires no training set, save the two images under comparison.

It is normal for pattern recognition techniques to make specialprovision for scale differences between the patterns being classified.Such differences in the visual domain can arise because of perspectiveand other factors and it is common for prior knowledge to beincorporated in the classifier to compensate. This is not necessary inthis method, provided that ε is not too large.

Another advantage is the ability to identify patterns embedded withinlarger data structures without suffering from exponentially growingcomputational demands or the effects of noise. This method therefore hasapplication to the problem of detecting copyright infringement whereportions of material have been cropped from larger works (of art, forexample), and the task of reducing the size of databases where it isknown that duplication is prevalent. It also has direct application tothe problem of identifying scene-of-crime fingerprints where only a partof the total print is available for matching. In the case of facialrecognition the method lends itself to searches based upon restrictedportions of the unknown face. This means, for example, that searches canbe carried out purely on the basis of the eye and nose region in caseswhere beards and moustaches might lead to ambiguity.

Unless provision is made for specific pattern content, the standardtemplate approaches to pattern recognition fail when the patterns undercomparison differ because of local distortions or small movements, aswould be the visual case with rustling trees, moving clouds, changes infacial expression, scene-of-crime fingerprints on irregular surfaces, ornoise, for example. Such provision requires prior knowledge of theapplication and will still cause the system to fail if the unseenpattern distortions do not conform to the system design requirements.This method is able to ignore the effects of local distortions withoutprior knowledge of the type of distortion.

The method may be applied to patterns of any dimension, such asone-dimensional audio signals, three dimensional video data (x,y,time),or n-dimensional time dependent vectors derived from any source such assensor arrays. In the case of speech recognition it is able to handlevariations in the speed of the speech without the use of specialheuristics. A conventional approach uses Dynamic Time Warping toovercome this problem, but invokes greater computational effort and thedanger of the warping process leading to increased misclassificationsespecially in a large multi-class problem. Furthermore portions ofutterances would be sufficient for word identification using thisinvention if they were unique in the domain of discourse (e.g. ‘yeah’instead of ‘yes’, missing the final sibilant).

This method deals with these problems by measuring the score ofneighbourhoods in each image. Whereas a picture element is importantwithin a single image if its neighbourhood is significantly differentfrom most other parts of the same image (as discussed in our earlierpatent applications mentioned above), it is cognitively important if itsneighbourhood is significantly similar to that in a neighbourhood in thesecond image. The effects of mis-registration and local distortions areconsiderably reduced as the scores are not affected within certainlimits.

The scores can be used to cluster groups of patterns possessing highscores relative to each other. Representative patterns taken from eachcluster may themselves be clustered to form super-clusters and theprocess continued to structure very large pattern databases.Query-By-Example retrieval can be carried out by measuring the scores toeach of the top level representative patterns and then to each of therepresentative patterns in sub-clusters corresponding to the highestscoring representative pattern in the previous cluster. It is likelythat some representative patterns will represent overlapping clusterscontaining patterns which happen to possess high scores with thoserepresentative patterns. The method may be used in combination with avisual attention mechanism to rapidly define an area of interest in thepattern and then derive scores between this restricted area and a set ofreference patterns (B patterns). In an aviation context a flying objectmight be quickly detected using visual attention and subsequentlycomputed scores would reveal whether the object was likely to be a birdor a plane.

Disparity Detection

Standard approaches to disparity detection rely heavily upon accurateregistration between two images so that the subtraction (which may becarried out piecewise for a small areas of the total image) takes placebetween picture elements that correspond to the same points on theoriginal object pictured in the two images. The resultingdifference-image highlights those areas that correspond to differencesin the original images. This becomes extremely difficult if noise ispresent as uncertainty is introduced into the estimate of the correctregistration position and many spurious differences can be generated asa result.

Even if noise is absent local distortions or slight subject movementswill cause mis-registration and areas of difference will be highlightedwhich are of little interest unless the distortion or movement itself isbeing measured. Linear or non-linear digital image registrationtechniques prior to subtraction partially compensates but does noteliminate this problem in a large proportion of cases [4].

A severe problem faced by conventional methods arises from the differentconditions under which the two images were created. Such differences maystem from the lighting, weather conditions, a slightly differentviewpoint, a different make of film, or a different camera with adifferent optical system. These interferences all contribute towards thegeneration of spurious differences between the images that do not relateto the content.

Differences between the two images caused by the conditions of imagecapture will be factored out by this method because such differenceswill affect the score equally across all parts of the image and will notdisturb the rank ordering of the individual scores. Significantillumination differences between the two images are compensated byemploying a larger threshold for the picture element matching (δ_(j) seeabove). This would be necessary, for example, if X-ray exposure timeswere different between the two images.

This method also has the advantage of being able to detect multipledisparities in which image A is compared with images B1,B2,B3 etc. Thiswould be useful in the case of CCTV intruder detection in which imageframes B1,B2,B3 etc would be typical examples of different atmosphericconditions and other normal background states, and an alarm would beraised only if disparities were detected in all the normal image frames.

It will be seen that these methods have relevance to almost allapplications requiring pattern recognition:

Face recognition, fingerprint recognition, OCR, image retrieval,trademark identification, forgery detection, surveillance, medicaldiagnosis, and others.

They are particularly relevant to the medical industry where thecomparison of very variable and noisy images is necessary to track theprogress of medical conditions. For example, the identification oflikely cancerous growth in mammograms is often gleaned from X-rays takenat different times and any automatic assistance would increaseradiologist's throughput. In another example, contrast enhancement isnormally obtained by subtracting the X-ray image of the normal breastfrom one taken after contrast injection and obtaining properregistration is not only difficult to obtain but is critical to thesuccess of the process. The methods we have described can provide aclearer result without the attendant registration problems.

Advanced software detection systems for mammograms (eg.http://www.r2tech.com/prd/) are quite capable of spotting specificfeatures commonly associated with cancer (eg microcalcifications), butthe problem of temporal comparisons is largely not addressed.

REFERENCES

[1] Vailaya A et al., Image Classification for Content-Based Indexing,IEEE Trans on Image Processing, Vol 10, No 1, pp 117-130, January 2001.

[2] Santini S & Jain R, Similarity Matching, in Proc 2^(nd) Asian Confon Computer Vision, pages II 544-548, IEEE, 1995.

[3] IEEE Trans PAMI—Special Section on Video Surveillance, vol 22 No 8,August 2000.

[4] Brown L G, A survey of image registration techniques, ACM ComputingSurveys, Vol. 24, No. 4 (December 1992), pp. 325-376.

1. A method of comparing a first pattern represented by a first orderedset of elements each having a value with a second pattern represented bya second ordered set of element each having a value, comprisingperforming, for each of a plurality of elements of the first ordered setthe steps of: (i) selecting from the first ordered set a plurality ofelements in the vicinity of the element under consideration, theselected elements having, within the ordered set, respective mutuallydifferent positions relative to the element under consideration; (ii)selecting an element of the second ordered set; (iii) comparing theselected plurality of elements of the first ordered set with a likeplurality of elements of the second ordered set each of which has withinthe second ordered set the same position relative to the selectedelement of the second ordered set as a respective one of the selectedplurality of elements of the first ordered set has relative to theelement under consideration, said comparison comprising comparing thevalue of each of the selected plurality of elements of the first orderedset with the value of the correspondingly positioned element of the likeplurality of elements of the second ordered set in accordance with apredetermined match criterion to produce a decision that the pluralityof elements of the first ordered set matches the plurality of elementsof the second ordered set; (iv) repeating said comparison wherein atleast some comparisons are made with a fresh selection of the pluralityof elements of the first ordered set and at least some comparisons aremade with a fresh selection of an element of the second ordered set; and(v) generating for the element under consideration a similarity measureas a function of the number of comparisons for which the comparisonindicates a match.
 2. A method according to claim 1 in which theplurality of elements of the first ordered set includes the elementunder consideration.
 3. A method according to claim 1 in which theselection of the plurality of elements of the first ordered set israndom or pseudo-random.
 4. A method according to claim 1 in which theselection of an element of the second ordered set is random orpseudo-random.
 5. A method according to claim 1 in which, following acomparison resulting in a match decision, a further comparison isperformed with a fresh selection of the plurality of elements of thefirst ordered set and the same selection of an element of the secondordered set.
 6. A method according to claim 1 in which, following acomparison resulting in a decision that there is no match, a furthercomparison is performed with a fresh selection of an element of thesecond ordered set and the same selection of the plurality of elementsof the first ordered set.
 7. A method according to claim 1, in whicheach value comprises a plurality of components.
 8. A method according toclaim 1, in which the comparison is performed a predetermined number oftimes.
 9. A method according to claim 1, in which the similarity measureis a function of the number of comparisons for which the comparisonindicates a match, excluding an initial number of comparisons.
 10. Amethod according to claim 1 in which, when an element underconsideration has resulted in a similarity measure indicative of a largenumber of matches and a further element of the first ordered set is tobe considered, the first selection of an element of the second orderedset for comparison therewith is of that element of the second orderedset whose position, within the second ordered set, relative to anelement of the second ordered set giving rise to said large number ofmatches, is the same as the position of the farther element of the firstordered set relative to the previous element under consideration.